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1.0  INTRODUCTION 

Analysis  of  the  prognostic  capabilities  of  an  acoustic  predictive  system  is  an  involved  task  which,  given 
resource  constraints,  require  that  the  analysis  of  the  system  be  limited  and  directed  toward  the  model’s 
intended  applications.  In  this  report,  the  capabilities  of  the  FeyRay  acoustic  prediction  model  (version  1.12 
Foreman  2007)  will  be  analyzed  through  comparison  with  the  Navy  Standard  Parabolic  Equation  Model 
(NSPE),  and  the  GRAB  (Gaussian  Ray  Bundle  eigenray  propagation  model)  in  a  limited  number  of 
environments.  Measurements  of  the  relationship  between  GRAB  and  NSPE  are  designed  to  quantify  the 
meaning  of  our  FeyRay  measurements  in  terms  of  another  commonly  used  OAML  model  for  Gaussian 
Beam  propagation. 

Each  of  the  models  is  exercised  on  system  where  the  results  are  known  to  match  the  results  of  the  standard 
output  cases  (for  GRAB  and  NSPE),  or  that  delivered  by  the  developer  (FeyRay).  The  FeyRay,  GRAB,  and 
PE  executions  were  made  on  a  Sun  1.8  GHz  Opteron  system  running  Debian  3.1  Linux,  with  the  models 
compiling  using  the  GCC  compiler  suite.  Secondary  runs,  to  ensure  quality  control,  were  made  on  a  Apple 
Macintosh  G4  computer  using  Absoft  C++/Fortran95  compilers. 

1.1  Scope 

The  acoustic  analysis  contained  in  this  report  is  based  on  comparing  the  output  of  FeyRay  and  GRAB  to  an 
accepted  acoustic  benchmark,  NSPE.  Accepting  NSPE  as  benchmark  does  not  imply  that  the  output  of  the 
NSPE  is  invariably  correct,  but  recognizes  NSPE  as  a  reasonable  standard  to  compare  against.  The  imposed 
constraints  limit  the  number  of  cases  to  be  analyzed  in  any  depth  to  a  small  number,  in  this  case  7.  But  by 
analyzing  a  large  number  of  cases  in  a  strictly  statistical  manner  some  conclusions  about  the  overall 
behavior  of  the  FeyRay  model  with  respect  to  its  peers  will  also  be  obtained. 

Each  of  the  models  was  run  in  the  mode  indicated  in  the  documentation  to  obtain  optimal  results.  The  ray 
fan  selection  in  GRAB  was  set  to  -89°  to  89°  in  0.1°  increments,  while  the  beam  fan  selection  was  made 
internally  by  FeyRay.  GRAB  was  executed  using  both  version  2  and  3,  with  the  optimal  result  selected. 

In  section  2  of  this  report  the  statistical  metrics  used  to  compare  the  model  predictions  are  introduced.  In 
section  3,  the  one  dimensional  sound  speed  fitting  capabilities  of  FeyRay  are  summarized,  the  performance 
of  FeyRay  and  GRAB  are  shown  against  the  know  Lloyd’s  mirror  result,  and  the  performance  of  the 
FeyRay  and  GRAB  predictions  against  a  number  of  NSPE  results  in  randomly  selected  environments  are 
shown.  Section  4  shows  the  modeling  capabilities  in  deep  water  and  section  5  shows  the  capabilities  in 
shallow  water.  Section  6  contains  the  conclusions. 


2.0  METRICS 

Measuring  the  difference  between  predictions  made  by  different  software  systems  can  be  done  in  a  number 
of  ways.  The  question  that  arises,  or  that  should  arise,  is  what  is  the  natural  way  to  measure  the 
performance  of  a  prediction  system?  A  transmission  loss  prediction  system  predicts  the  relative  acoustic 
energy  as  a  function  of  range.  In  the  field  it  is  not  possible  to  measure  transmission  loss  directly.  In  fact, 
one  measures  received  levels,  then  computes  transmission  losses  from  a  calibrated  source.  As  such,  a  sonar 
system  operator  does  not  detect  transmission  loss,  rather,  the  operator  detects  a  signal  in  excess  of  the 
background  ambient  level.  That  is,  detected  signals  exceed  some  threshold  level.  As  such,  the  measurement 
criteria  of  effectiveness  of  a  prediction  system  should  be  based  on  the  ability  of  the  prediction  system  to 
predict  ranges  at  which  the  transmission  losses  are  low  enough  to  allow  detection. 

A  number  of  computational  issues  arise  that  must  be  dealt  with  on  an  individual  basis.  While  each  of  these 
issues  has  effects  on  the  predictive  capability  of  a  model,  most  are  not  seen  when  viewing  the  product  of  an 
acoustic  simulation.  Experience  with  the  testing  of  other  Gaussian  ray  models  for  OAML  indicates  that 
three  factors  are  of  critical  importance: 
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•  the  sound  speed  fitting  capability  of  the  model, 

•  accuracy  of  the  model  in  the  Lloyds  mirror  scenario,  and 

•  the  ability  of  the  model  to  cope  with  a  wide  variety  of  input  (robustness). 


Thresholds  were  defined  in  terms  of  the  Figure  of  Merit  needed  to  detect  three  types  of  passive  sonar 
targets: 


•  Strong  Target  =  100  dB 

•  Medium  Target  =  80  dB 

•  Weak  Target  =  60  dB 


Transmission  loss  values  below  the  100  dB  level  were  assumed  to  be  tactically  insignificant. 


Three  statistics  are  defined  to  measure  the  effectiveness  of  the  performance  prediction.  First,  the  difference 
of  each  model  from  the  reference  solution  is  measured  and  averaged. 


where: 


AL  =  d-£TL(rn)-PE(r„) 

n=l 


rn  =  series  of  ranges  at  which  TL  is  calculated. 

TL  =  transmission  loss  of  model  under  investigation 

PE  =  transmission  loss  of  NSPE  model  at  the  same  range, 

AL  =  average  difference  in  level  (smaller  is  better). 

In  order  to  focus  on  tactically  significant  results,  this  calculation  excludes  any  range  where  the  NSPE 
solution  dips  below  the  100  dB  threshold.  Measurements  are  made  in  decibel  space  rather  than  in  linear 
units  to  measure  the  average  effect  of  differences  on  signal  excess  and,  in  turn,  sonar  performance. 

The  second  metric  is  a  correlation  error  based  on  the  Pearson  product-moment  correlation  coefficient. 


=  1  -  r  =  1  - 


G  VG, 


where 


r  =  the  Pearson  product-moment  correlation  coefficient, 

Gxy  =  the  covariance  of  the  sequences  x  and  y, 

Gx  =  the  standard  deviation  of  x, 

Gy  =  the  standard  deviation  of  y,  and 

Terror  =  the  average  correlation  error  (smaller  is  better). 

The  correlation  error  measures  differences  in  the  fluctuations  between  models.  It  is  designed  to  provide  a 
measure  of  model-to-model  equivalence  that  is  independent  of  the  average  error  in  level.  Correlation  error 
was  chosen  instead  of  correlation  as  a  metric  so  that  each  metric  specifies  improved  equivalence  to  the 
NSPE  as  values  decrease.  Measurements  are  made  in  decibel  space  rather  than  in  linear  units  to  measure 
the  average  effect  of  fluctuations  on  signal  excess  and,  in  turn,  sonar  performance. 

The  last  metric  is  an  estimate  of  detection  range  for  the  three  types  of  targets  specified  above.  Tabulated 
values  will  be  presented  which  illustrate  the  ranges  at  which  detection  first  ceases  to  occur,  and  last  ceases 
to  occur  are  accumulated. 
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3.0  COMPONENTS  TESTS 

These  tests  were  designed  to  analyze  individual  features  of  the  model. 


3. 1  Sound  Speed  Fitting 

FeyRay  requires  that  the  sound  speed  interpolant,  and  its  first  3  derivatives,  be  continuous.  The  FeyRay 
sound  speed  algorithm  is  a  two  part  construction.  The  algorithm  is  a  tension  spline  interpolant  in  the 
vertical  (a  global  fit  along  a  single  sound  speed  profile),  coupled  with  a  degenerate  firth  order  polynomial 
in  the  horizontal  between  two  adjacent  vertical  profiles  (a  local  method  in  the  horizontal). 

The  fit  of  the  interpolant  to  the  input  sound  speed  field  was  examined  using  a  Munk  profile  (Munk  1974) 
using  the  parameters  found  in  Porter  and  Bucker  (1987).  A  source  is  placed  at  1000m  depth  ,  a  receiver  at 
800m  depth,  and  beams  are  launched  from  -14.66  to  14.66  degrees  with  a  spacing  of  0.02  degrees,  and  0.01 
degrees.  At  each  call  in  the  ray  tracing  algorithm  where  the  sound  speed  is  updated  the  value  of  the  depth 
and  the  sound  speed  is  intercepted.  Figure  1  shows  a  plot  of  the  sound  speed  interpolant  versus  depth  for 
the  two  different  spacing.  The  two  set  of  values  define  the  Munk  profile  to  a  very  high  degree. 

There  are  small  differences  between  the  values  derived  from  the  analytic  Munk  expression,  and  the  FeyRay 
derived  interpolant.  Figure  2  shows  a  plot  of  the  difference  between  the  FeyRay  interpolant  and  the  true 
value.  The  interpolant  varies  from  the  true  value  by  less  than  0.001  per  cent,  which  is  sufficiently  accurate 
for  acoustic  purposes.  However,  there  is  an  aspect  of  the  fit  that  must  be  mentioned.  Since  the  fit  oscillates 
between  negative  and  positive  variations,  this  implies  that  higher  derivatives  also  oscillate.  The  magnitude 
of  the  oscillations  will  be  small,  but  may  be  visible  when  the  eigenrays  from  a  fine  input  beam  spacing  are 
compared  to  the  eigenrays  from  a  coarse  input  beam  spacing. 
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Figure  1.  Plot  of  Munk  sound  speed  obtained  from  FeyRay  interpolation  for  two  different  beam  fans. 
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Sound  speed  Deviation  (m/s) 

Figure  2.  Plot  of  difference  between  analytic  Munk  Profile  and  the  FeyRay  interpolant  using  two  different 
beam  samplings. 


3.2  Lloyds  Mirror 

The  Lloyd’s  mirror  test  has  been  used  primarily  as  a  means  of  testing  both  the  reflection  from  a  totally 
reflecting  surface,  and  the  wide  angle  capabilities  of  parabolic  approximation  models.  Since  wide  angle 
capability  is  not  a  problem  for  beam  models,  the  Lloyd's  mirror  test  was  used  to  demonstrate  the  reflection 
capability  and  the  interpolation  components  of  the  FeyRay  and  GRAB  models.  In  theory  a  Gaussian  beam 
model  gains  much  of  its  computational  advantage  (over  classic  ray  theory  constructs)  in  its  ability  to 
properly  model  caustics.  Gaussian  beam  models  also  support  the  ability  to  model  large  swatches  of  low 
environmental  variability  with  a  very  few  widely  spaced  beams.  The  Lloyd's  mirror  scenario  used  for  this 
test  is  taken  from  the  PE  Workshop  II  specifications.  The  water  depth  was  set  to  5000  m,  the  source  depth 
to  350  meters,  and  the  receiver  depth  to  3990  meters.  The  frequency  was  40  Hz.  Since  bottom  returns  are 
not  a  feature  supported  by  this  test,  the  bottom  boundary  condition  for  both  models  was  set  to  a  density 
ratio  of  1.0,  a  sound  speed  ratio  of  1.0,  attenuation  was  set  to  0.001  dB  per  wavelength  to  create  a  non¬ 
reflecting  interface.  Both  models  were  run  using  default  beam/ray  fans.  For  GRAB  this  meant  that  the  rays 
were  specified  from  -89°  to  89°  in  0.10°  increments.  FeyRay  internally  selected  its  beams  for  this  case. 
Figure  3  shows  the  benchmark  solution,  the  GRAB  solution  and  the  FeyRay  solution  for  ranges  up  to  10 
km. 
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Figure  3  Plot  of  Lloyd's  mirror  transmission  loss  showing  image  solution,  FeyRay  solution,  and 
GRAB  solution. 

The  FeyRay  and  GRAB  solutions  agree  with  the  benchmark  solution  for  ranges  beyond  600  meters.  The 
difference  at  short  ranges  is  best  understood  as  an  artifact  of  using  the  default  launch  angles.  Better 
performance  should  be  expected  at  short  ranges  for  both  models  if  launch  angles  were  changed  to  improve 
the  sampling  of  high  angle  paths. 


3.3  Robustness  Test 

The  robustness  of  the  models  was  tested  by  computing  transmission  losses  for  one  thousand  twenty  three 
distinct  environments  randomly  selected  from  Norwegian,  North  Atlantic,  Mediterranean,  Western  and 
Eastern  Pacific  areas.  The  seasons  were  selected  such  that  spring,  summer,  fall,  and  winter  were 
equiprobable.  Surface  loss  was  assumed  to  be  zero.  The  model  was  exercised  for  four  frequencies  (50,  100, 
250,  and  2500  Hz)  with  one  source  depth,  and  three  receiver  depths  for  each  test  case.  Each  model  was  run 
with  default  beam/ray  fan  specifications  (see  the  Lloyds  mirror  specification).  NSPE  solutions  were  used  as 
the  benchmark  solution. 

Each  run  was  made  as  a  stand  alone  process  (as  opposed  to  a  library  process)  running  in  a  batch  file.  Thus 
each  run  loaded  a  fresh  executable  in  memory,  and  cleared  memory  upon  completion.  The  robustness  of  the 
model  was  assessed  by  investigating  whether  each  test  case  ran  to  normal  completion  with  no  errors  issued 
by  the  program  or  operating  system.  The  results  are  summarized  in  the  set  of  figures  below.  The  figures 
consist  of  histograms  of  distribution  of  the  correlation  error,  and  average  error  for  each  model  relative  to 
the  benchmark  for  strong  and  medium  targets. 

Figure  4  shows  the  distribution  of  correlation  error  for  FeyRay  using  a  100  dB  threshold,  and  Figure  5 
shows  the  distribution  of  correlation  error  for  GRAB  using  a  100  dB  threshold.  The  difference  in  the  mean 
correlation  error  is  small  (0.06)  and  the  difference  in  standard  deviations  is  small  (0.016)  with  FeyRay  in 
demonstrating  a  smaller  mean  correlation  error,  and  GRAB  demonstrating  a  slightly  tighter  distribution. 
Figure  6  shows  the  distribution  of  average  error  for  FeyRay  using  a  100  dB  threshold,  and  Figure  7  shows 
the  distribution  of  average  error  for  GRAB  using  a  100  dB  threshold.  FeyRay  has  a  smaller  average  error 
than  GRAB  (0.24  dB  versus  -4.79dB)  with  FeyRay  predicting  lower  signal  excess  than  the  benchmark,  and 
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GRAB  predicting  higher  signal  excess.  GRAB  distribution  is  more  compact  than  the  FeyRay  distribution 
(2.79  dB  versus  3.36  dB). 

Figure  8  shows  the  distribution  of  correlation  error  for  FeyRay  using  a  80  dB  threshold,  and  Figure  9  shows 
the  distribution  of  correlation  error  for  GRAB  using  a  80  dB  threshold.  The  difference  in  the  mean 
correlation  error  is  0.109  and  the  difference  in  standard  deviations  is  small  (0.029)  with  FeyRay  in 
demonstrating  a  smaller  mean  correlation  error,  and  GRAB  demonstrating  a  slightly  tighter  distribution. 

The  difference  in  the  correlation  error,  and  standard  deviation  is  larger  for  the  80  dB  threshold  although 
they  the  ratio  of  the  numbers  is  approximately  the  same.  Figure  10  shows  the  distribution  of  average  error 
for  FeyRay  using  a  80  dB  threshold,  and  Figure  1 1  shows  the  distribution  of  average  error  for  GRAB  using 
a  80  dB  threshold).  FeyRay  has  a  smaller  average  error  than  GRAB  (-0.48  dB  versus  -1.76  dB)  with  both 
model  predicting  higher  signal  excess  than  the  benchmark  FeyRay  distribution  is  more  compact  than  the 
GRAB  distribution  (0.80  dB  versus  1.54  dB). 


Correlation  Error 

Figure  4  Histogram  of  correlation  error  for  FeyRay  robustness  test  for  100  dB  threshold. 
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Figure  5  Histogram  of  correlation  error  for  GRAB  robustness  for  100  db  threshold. 
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Figure  6  Histogram  of  average  error  for  FeyRay  Robustness  test  using  100  dB  threshold. 
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Figure  7  Histogram  of  average  error  for  GRAB  robustness  test  using  100  dB  threshold. 
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Figure  8  Histogram  of  correlation  error  for  FeyRay  robustness  test  using  80  dB  threshold. 
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Figure  9  Histogram  of  correlation  error  for  GRAB  robustness  test  using  80  dB  threshold. 
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Figure  10  Histogram  of  average  error  for  FeyRay  robustness  test  using  80  dB  threshold. 
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Figure  11  Histogram  of  average  error  for  GRAB  robustness  test  using  80  dB  threshold. 
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3.4  Execution  Speed 

Since  improved  computational  speed  is  one  of  the  features  touted  by  the  FeyRay  developer,  timing 
comparisons  were  also  performed  on  the  GRAB  and  FeyRay  models.  Each  of  the  models  was  modified 
slightly  so  that  the  time  to  compute  transmission  loss  could  be  logged  without  including  the  time  needed  to 
load  the  program  into  memory.  The  time  required  to  compute  the  transmission  loss  for  each  of  the 
robustness  tests  was  recorded  and  averaged.  In  each  case,  results  and  temporary  files  were  written  to  a 
RAM  disk  to  minimize  the  impact  of  disk  I/O  differences  between  the  models.  Note  that  these  results  are 
for  the  calculation  of  complete  transmission  loss  curves  from  0-140  km  with  output  every  1/3  km. 

GRAB  comes  in  two  flavors  on  the  distribution  disk:  version  2  and  version  3.  GRAB  version  2uses  a 
triangular  interpolation  scheme  for  the  sound  velocity  profile.  GRAB  version  3  uses  a  less  accurate  bi¬ 
linear  interpolation  that  greatly  improves  the  execution  speed  of  GRAB.  In  the  timing  tests,  both  versions 
of  GRAB  were  used.  In  the  rest  of  this  document,  the  more  accurate  version  of  GRAB  was  used. 

These  timing  results  were  collected  as  part  of  the  robustness  tests  described  in  Section  3.3.  Each  test  case 
had  4  sound  speed  profiles,  140  km  range,  range  dependent  bathymetry  (2500-5000  m),  2  bottom  loss 
provinces  and  the  default  ray  trace  parameters  Table  1  shows  the  performance.  The  run  times  reported  here 
are  for  a  specific  computer,  operating  system,  and  compiler  suite  as  well  as  the  type  of  test  case  reported. 
Variations  in  performance  ratios  should  be  expected,  as  usage  of  cache,  real  and  virtual  memory  are 
different  for  each  of  these  models. 


Table  1  Relative  run  time  performance  of  FeyRay  and  GRAB. 


Candidate 

Model 

Average 

Standard 

Deviation 

Ratio 

(Model/FeyRay) 

GRAB  V2 

145.204 

1.384 

20.82 

GRAB  V3 

35.818 

1.214 

5.14 

FeyRay 

6.975 

0.122 

1.00 
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4.0  DEEP  WATER  ACCURACY  TESTS 

The  deep  water  accuracy  analysis  focuses  on  areas  where  features  in  the  sound  speed  profile  are  paramount. 
Three  deep  cases  will  now  be  considered.  These  cases  are  one  using  a  range  independent  bottom  with  a 
range  independent  Munk  profile  (as  illustrated  in  Figure  1),  a  range  dependent  case  with  a  surface  duct 
overlying  a  sound  channel,  and  a  range  dependent  case  with  a  double  duct  capped  by  a  surface  duct. 


4. 1  Munk  Profile 

A  simple  test  of  model  performance  prediction  is  a  convergence  zone  in  deep  water.  The  source  is  placed  at 
1000  m,  the  receiver  at  800  m,  the  water  depth  is  set  to  5000  m.  The  bottom  is  defined  as  in  the  Lloyd’s 
mirror  case  (density  ratio  1.0,  sound  speed  ratio  1.0,  and  attenuation  as  0.001  dB  per  wavelength).  The 
sound  speed  is  depicted  in  Figure  1.  The  default  ray  fans  for  GRAB,  and  beam  fans  for  FeyRay  are 
selected.  Propagation  range  is  set  to  100  km. 

Figure  12  shows  the  prediction  of  the  PE  model,  FeyRay,  and  GRAB  as  functions  of  range  for  a  frequency 
of  25  Hz.  Both  GRAB  and  FeyRay  track  the  PE  solution  through  the  first  CZ  when  GRAB  shows  a 
tendency  to  under-predict  the  transmission  loss.  Figure  13  shows  the  transmission  loss  prediction  of  the  PE 
model,  FeyRay,  and  GRAB  as  functions  of  range  for  a  frequency  of  100  Hz.  The  structure  of  the 
predictions  are  similar,  except  that  FeyRay  suffers  a  large  over-prediction  of  loss  between  the  ranges  of  20 
and  35  km.  Figure  14  shows  the  prediction  of  the  PE  model,  FeyRay  and  GRAB  as  functions  of  range  for  a 
frequency  of  250  Hz.  At  250  Hz  FeyRay  exhibits  the  same  over-prediction  of  transmission  loss  between  the 
ranges  of  20  and  35  km  as  in  the  100  Hz  case.  GRAB  is  tracking  the  null  beyond  the  first  CZ  properly. 
Figure  15  shows  the  predictions  of  the  PE  model,  FeyRay,  and  GRAB  as  functions  of  range  for  a  frequency 
of  2500  Hz.  GRAB  and  FeyRay  continue  to  produce  similar  outputs  to  the  250  Hz  case,  with  the  same 
structure. 

Table  2  shows  the  correlation  error  ,  average  error,  and  maximum  detection  range  in  km  for  FeyRay  and 
GRAB  for  a  60  dB  threshold.  This  table  includes  the  maximum  detection  range  for  PE.  The  ranges  are 
short,  thus  the  number  of  data  points  that  go  into  the  error  and  correlation  error  are  small.  Both  FeyRay, 
and  GRAB  predict  the  PE  solution  well. 


Table  2  Correlation  error,  average  error,  and  maximum  detection  range  for  FeyRay,  and  GRAB  for 
the  Munk  profile  range  independent  case  for  60  dB  threshold. 


Freq. 

FeyRay 
corr.  Err 

FeyRay 

error 

GRAB 

corr  err 

GRAB 

error 

Max  range 
PE 

Max  range 
FeyRay 

Max  range 
GRAB 

25 

0% 

0.0  dB 

0% 

0.0  dB 

1  km 

1  km 

1  km 

100 

4% 

0.2  dB 

19% 

0.4  dB 

1  km 

2  km 

1  km 

250 

0% 

0.0  dB 

1  % 

0.2  dB 

2  km 

2  km 

2  km 

2500 

3  % 

0.2  dB 

2% 

0.2  dB 

2  km 

1  km 

1  km 
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Table  3  shows  the  correlation  error  and  average  error  for  FeyRay  and  GRAB  for  an  80  dB  threshold.  The 
correlation  error  for  both  models  is  small,  and  the  average  error  is  under  0.5  dB  for  all  frequencies. 


Table  3  Correlation  error  and  average  error  for  FeyRay  and  GRAB  for  the  Munk  profile  range 
independent  case  or  80  dB  threshold.. 


Frequency 

FeyRay 

Correlation  Error 

FeyRay  error 

GRAB  Correlation 
Error 

GRAB  error 

25 

1  % 

0.1  dB 

2% 

0.0  dB 

100 

10% 

0.1  dB 

11  % 

0.0  dB 

250 

12% 

0.0  dB 

15  % 

0.0  dB 

2500 

9% 

0.4  dB 

17% 

0.2  dB 

Table  4  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  an  80  dB 
threshold.  FeyRay  and  GRAB  show  minimum  and  maximum  ranges  consistent  with  PE,  except  for  the 
2500  Hz  case  which  appears  anomalous. 


Table  4  Minimum  and  maximum  range  of  detection  ranges  (in  km)  for  PE,  FeyRay  and  GRAB  for 
an  80  dB  threshold  for  the  Munk  profile  case  (maximum  range  is  100  km) 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

2.5  km 

2.5  km 

2.5  km 

56.5  km 

12.5  km 

61  km 

100 

8.5  km 

8.5  km 

6.5  km 

59.5  km 

54  km 

55.5  km 

250 

3  km 

10  km 

8.5  km 

58  km 

40.5  km 

56.5  km 

2500 

8.5  km 

3  km 

3  km 

89  km 

12  km 

12  km 

Table  5  shows  the  correlation  error  for  FeyRay  and  GRAB  for  an  100  dB  threshold.  The  correlation  error 
for  both  models  is  small,  and  the  average  error  is  under  2.0  dB  for  all  frequencies  except  the  possibly 
anomalous  2500  Hz  case. 


Table  5  Correlation  error  for  FeyRay  and  GRAB  for  100  dB  threshold  case  for  the  Munk  profile.. 


Frequency 

FeyRay 

Correlation  Error 

FeyRay  error 

GRAB  coefficient 
of  error 

GRAB  error 

25 

7  % 

1.0  dB 

10% 

-0.7  dB 

100 

10% 

1.1  dB 

13  % 

0.1  dB 

250 

12% 

1.7  dB 

15  % 

-0.7  dB 

2500 

18  % 

9.0  dB 

25  % 

7.7  dB 

Table  6  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  an  100  dB 
threshold.  FeyRay  and  GRAB  show  minimum  and  maximum  ranges  consistent  with  PE 


Table  6  Minimum  and  maximum  detection  ranges  (in  km)  for  PE,  FeyRay,  and  GRAB  for  an  100  dB 
threshold  for  the  Munk  profile  case  (maximum  range  is  100  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

14.5  km 

14.5  km 

14.5  km 

all 

all 

all 

100 

13  km 

13.5  km 

15  km 

all 

all 

all 

250 

12.5  km 

15  km 

15  km 

all 

all 

all 

2500 

14.5  km 

15  km 

12.5  km 

all 

90  km 

99  km 
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Figure  12  Plot  of  transmission  loss  for  PE,  FeyRay,  and  GRAB  for  the  range  independent  Munk 
profile  for  a  frequency  of  25  Hz. 
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Figure  13  Plot  of  transmission  loss  for  PE,  FeyRay,  and  GRAB  for  the  range  independent  Munk 
profile  for  a  frequency  of  100  Hz. 
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Figure  14  Plot  of  transmission  loss  for  PE,  FeyRay  and  GRAB  for  the  range  independent  Munk 
profile  case  for  a  frequency  of  250  Hz. 
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Figure  15  Plot  of  transmission  loss  for  PE,  FeyRay  and  GRAB  for  the  range  independent  Munk 
profile  case  for  a  frequency  of  2500  Hz. 


4.2  Surface  Duct  over  Deep  Duct 

The  surface  duct  over  deep  duct  test  case  (also  referred  to  as  wlan069)  features  a  source  depth  of  25.4 
meters  and  a  receiver  depth  of  385.1  meters.  Figure  16  depicts  the  environment.  The  bottom  for  this  case  is 
made  a  constant  silty-clay  (density  1.488,  sound  speed  1.014,  attenuation  0.1  dB  per  wavelength).  Analysis 
of  the  environment  shows  that  the  source  is  located  within  the  surface  duct,  and  the  receiver  below  the  base 
of  the  surface  duct.  There  is  some  variability  in  the  location  and  strength  of  the  surface  duct  as  a  function  of 
range,  thus  a  tensor  spline  approach  to  fitting  the  sound  speed  field  may  lead  to  problems.  FeyRay  and 
GRAB  are  both  exercised  using  their  default  ray/beam  fan  definitions. 

The  prediction  for  PE  model ,  FeyRay,  and  GRAB  for  25  Hz  are  show  in  Figure  17.  At  25  Hz  GRAB  has  a 
tendency  to  under-predict  the  transmission  losses  for  this  frequency.  Figure  18  shows  the  predictions  for  PE 
model,  FeyRay,  and  GRAB  at  100  Hz.  At  this  frequency  the  GRAB  model  more  clearly  follows  the  level  of 
the  PE  model,  while  FeyRay  is  over-predicting  the  transmission  loss  at  range.  Figure  19  shows  the  PE 
model,  FeyRay,  and  GRAB  predictions  for  250  Hz  at  a  function  of  range.  At  250  Hz  both  FeyRay,  and 
GRAB  predictions  follow  the  PE  solution,  except  for  a  minor  excursion  in  the  FeyRay  solution  near  90  km 
range.  Figure  20  shows  the  PE  model,  FeyRay,  and  GRAB  predictions  for  250  Hz  at  a  function  of  range 
The  2500  Hz  solution  for  PE  model ,  FeyRay,  and  GRAB  has  FeyRay  and  GRAB  agreeing  at  range  against 
PE.  In  this  scenario,  results  will  not  be  shown  for  the  60  dB  threshold  because  the  environment  is  such  that 
the  weak  target  is  rarely  detected. 
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Figure  16  Sound  speed  and  bathymetric  environment  for  surface  duct  over  deep  duct  test  case  (wlan 
069). 

Table  7  shows  the  correlation  error  for  FeyRay  and  GRAB  for  an  80  dB  threshold.  The  correlation  error  for 
both  models  is  small,  and  the  average  error  is  under  0.3  dB  for  all  frequencies.  The  overall  appearance  seen 
in  the  transmission  loss  plots  translates  into  excellent  statistical  fits. 


Table  7  Correlation  error  and  average  error  for  FeyRay  and  GRAB  for  the  surface  duct  over  deep 
channel  profile  range  dependent  case  (wlan069  extraction)  for  80  dB  threshold. 


Frequency 

FeyRay 

Correlation  Error 

FeyRay  error 

GRAB  coefficient 
of  error 

GRAB  error 

25 

2% 

-0.1  dB 

7  % 

-0.2  dB 

100 

3  % 

0.1  dB 

27  % 

-0.3  dB 

250 

7  % 

0.2  dB 

27  % 

0.0  dB 

2500 

20% 

0.2  dB 

21  % 

0.1  dB 
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Table  8  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  an  80  dB 
threshold.  FeyRay  and  GRAB  show  minimum  and  maximum  ranges  consistent  with  PE,  except  for  the 
2500  Hz  case  which  appears  anomalous. 


Table  8  Minimum  and  maximum  ranges  (in  km)  of  detection  for  PE,  FeyRay,  and  GRAB  for  the 
surface  duct  over  deep  channel  range  dependent  profile  case  (wlan069  extraction)  for  80  dB 
threshold  (maximum  range  is  140  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

5  km 

5.5  km 

3  km 

5.5  km 

13  km 

34  km 

100 

1 1.5  km 

7  km 

11  km 

68  km 

62  km 

89  km 

250 

3.5  km 

3.5  km 

7.5  km 

67  km 

68  km 

68  km 

2500 

6.5  km 

4.5  km 

4.5  km 

64  km 

8.5  km 

21  km 

Table  9  shows  the  correlation  error  for  FeyRay  and  GRAB  for  an  100  dB  threshold.  The  correlation  error  is 
still  small  for  both  GRAB  and  FeyRay  with  the  possible  exception  of  the  250  Hz  GRAB  run.  GRAB 
performs  excellently  as  measured  by  transmission  loss  except  at  25  Hz,  and  FeyRay  performs  excellently 
except  at  2500  Hz. 


Table  9  Correlation  error  for  FeyRay  and  GRAB  for  the  surface  duct  over  deep  channel  profile 
range  dependent  case  (wlan069  extraction)  for  100  dB  threshold. 


Frequency 

FeyRay 

Correlation  Error 

FeyRay  error 

GRAB  coefficient 
of  error 

GRAB  error 

25 

25  % 

-1.8  dB 

37  % 

-6.3  dB 

100 

39% 

3.1  dB 

39% 

-2.7  dB 

250 

37  % 

3.2  dB 

51  % 

-1.4  dB 

2500 

28  % 

7.3  dB 

34% 

3.8  dB 

Table  10  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  an  80  dB 
threshold.  FeyRay  and  GRAB  show  minimum  and  maximum  ranges  consistent  with  PE 


Table  10  Minimum  and  maximum  range  (in  km)  of  signal  excess  for  PE,  FeyRay  and  GRAB  for  the 
surface  duct  over  deep  channel  profile  range  dependent  case  (wlan069  extraction)  for  100  dB 
threshold  (maximum  range  is  140  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

12  km 

15.5  km 

26  km 

139  km 

all 

all 

100 

33.5  km 

12.5  km 

37.5  km 

all 

all 

all 

250 

32  km 

26  km 

12.5  km 

all 

all 

all 

2500 

28.5  km 

21.5  km 

13  km 

all 

84.5  km 

125.5  km 
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Figure  17  Plot  of  predicted  transmission  loss  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over  deep 
channel  profile  range  dependent  case  (wlan069  extraction)  for  a  frequency  of  25  Hz. 
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Figure  18  Plot  of  predicted  transmission  loss  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over  deep 
channel  profile  range  dependent  case  (wlan069  extraction)  for  a  frequency  of  100  Hz. 
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Figure  19  Plot  of  predicted  transmission  loss  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over  deep 
channel  profile  range  dependent  case  (wlan069  extraction)  for  a  frequency  of  250  Hz. 
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Figure  20  Plot  of  predicted  transmission  loss  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over  deep 
channel  profile  range  dependent  case  (wlan069  extraction)  for  a  frequency  of  2500  Hz. 


4.3  Surface  Duct  over  Double  Duct 

The  surface  duct  over  double  duct  test  case  (also  referred  to  as  wlanl83)  features  a  source  depth  of  25.4 
meters  and  a  receiver  depth  of  385.1  meters.  Figure  21  depicts  the  environment.  The  bottom  for  this  case  is 
made  a  constant  silty-clay  (density  1.488,  sound  speed  1.014,  attenuation  0.1  dB  per  wavelength).  Analysis 
of  the  environment  shows  that  the  source  is  located  within  the  surface  duct,  and  the  receiver  near  the 
channel  axis  of  the  first  deep  duct.  There  is  some  variability  in  the  location  and  strength  of  the  surface  duct 
as  a  function  of  range,  and  some  variability  in  the  depths,  and  strengths  of  the  deep  ducts.  Again  the  tensor 
spline  approach  to  fitting  the  sound  speed  field  may  lead  to  problems.  FeyRay  and  GRAB  are  both 
exercised  using  their  default  ray/beam  fan  definitions.  In  this  scenario,  results  will  not  be  shown  for  the  60 
dB  threshold  because  the  environment  is  such  that  the  weak  target  is  rarely  detected. 

Figure  22  shows  the  solutions  generated  by  PE,  FeyRay,  and  GRAB  for  this  test  case  at  a  frequency  of  25 
Hz.  The  solutions  show  FeyRay  tracking  the  PE  solution  quite  well,  while  GRAB  is  predicting  too  little 
loss  with  very  little  structure.  Figure  23  shows  the  solutions  generated  by  PE,  FeyRay  and  GRAB  for  this 
test  case  at  100  Hz.  At  this  frequency  GRAB  solution  is  closer  in  appearance  to  PE  solution  than  the 
previous  case.  The  FeyRay  solution  tends  to  track  the  PE  solution  by  the  levels  are  too  low.  The  250  Hz 
solutions  for  PE,  FeyRay  and  GRAB  are  shown  in  Figure  24.  The  trends  observed  at  25  Hz,  and  100  Hz 
continue.  There  is  a  tendency  for  GRAB  solutions  to  be  optimistic  at  the  mid-point  between  profiles,  and 
the  FeyRay  solutions  to  be  pessimistic.  The  2500  Hz  case  shown  in  Figure  25  for  PE,  FeyRay  and  GRAB 
shows  the  continuation  of  the  previous  trends  in  FeyRay.  That  is  at  the  mid-point  between  sound  speed 
fields  a  minimum  whose  effects  increases  with  frequency  occurs.  The  GRAB  tendency  to  underestimate 
loss  disappears. 
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Figure  21  Sound  speed  and  bathymetric  environment  for  surface  duct  over  double  duct  range 
dependent  case  (wlanl83  extraction). 


Table  11  shows  the  correlation  error  for  FeyRay  and  GRAB  for  an  80  dB  threshold.  The  correlation  error 
for  both  models  is  small,  and  the  average  error  is  under  0.2  dB  for  all  frequencies.  It  should  be  noted  that 
the  sample  for  this  estimate  is  small. 


Table  11  Correlation  error  and  average  error  for  FeyRay,  and  GRAB  for  the  surface  duct  over 
double  duct  profile  range  dependent  environment  (wlanl83  extraction)  for  80  dB  threshold. 


Frequency 

FeyRay 

Correlation  Error 

FeyRay  error 

GRAB  coefficient 
of  error 

GRAB  error 

25 

3  % 

-0.2  dB 

12% 

-0.2  dB 

100 

23  % 

0.0  dB 

26% 

-0.2  dB 

250 

16% 

-0.1  dB 

32% 

-0.2  dB 

2500 

12% 

0.2  dB 

12% 

0.0  dB 
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Table  12  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  an  80  dB 
threshold.  FeyRay  and  GRAB  show  minimum  and  maximum  ranges  consistent  with  PE,  except  for  the 
FeyRay  at  100  Hz,  and  GRAB  at  250  Hz  and  the  2500  Hz  case  which  appears  anomalous. 


Table  12  Minimum  and  maximum  detection  ranges  (in  km)  for  PE,  FeyRay,  and  GRAB  for  the 
surface  duct  over  double  duct  profile  range  dependent  environment  (wlanl83  extraction)  for  80  dB 
threshold  (maximum  detection  range  is  140  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

4  km 

4  km 

3.5  km 

5  km 

13  km 

54.5  km 

100 

0.5  km 

5.5  km 

1.5  km 

126  km 

9  km 

64  km 

250 

6.5  km 

5  km 

7  km 

63.5  km 

64  km 

125  km 

2500 

7  km 

5.5  km 

6.5  km 

63  km 

8  km 

8.5  km 

Table  13  shows  the  correlation  error  for  FeyRay  and  GRAB  for  an  100  dB  threshold.  In  general  FeyRay 
has  a  better  correlation  error  than  does  GRAB.  FeyRay  outperforms  GRAB  at  25  Hz,  GRAB  outperforms 
FeyRay  at  2500  Hz.  Between  those  two  frequencies  there  is  a  slight  edge  for  FeyRay. 


Table  13  Correlation  error  and  average  error  for  FeyRay,  and  GRAB  for  the  surface  duct  over 
double  duct  profile  range  dependent  environment  (wlanl83  extraction)  for  100  dB. 


Frequency 

FeyRay 

Correlation  Error 

FeyRay  error 

GRAB  coefficient 
of  error 

GRAB  error 

25 

20% 

-0.5  dB 

33  % 

-7.7  dB 

100 

34% 

3.0  dB 

51  % 

-3.5  dB 

250 

35  % 

3.3  dB 

55  % 

-4.6  dB 

2500 

33  % 

7.3  dB 

40% 

2.7  dB 

Table  14  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  a  100  dB 
threshold.  FeyRay  and  GRAB  show  minimum  and  maximum  ranges  consistent  with  PE 


Table  14  Minimum  and  maximum  detection  ranges  (in  km)  for  PE,  FeyRay,  and  GRAB  for  the 
surface  duct  over  double  duct  profile  range  dependent  environment  (wlanl83  extraction)  for  100  dB 
threshold  (maximum  detection  range  is  140  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

12  km 

15.0  km 

29.5  km 

134  km 

all 

all 

100 

12  km 

22.5  km 

22.5  km 

all 

139.5  km 

all 

250 

11  km 

15.5  km 

13.5  km 

all 

all 

all 

2500 

14  km 

8  km 

20  km 

all 

75  km 

130  km 

25 
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Figure  22  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over 
double  duct  range  dependent  environment  (wlanl83  extraction)  for  a  frequency  of  25  Hz. 
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Figure  23  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over 
double  duct  profile  range  dependent  environment  (wlanl83  extraction)  for  a  frequency  of  100  Hz. 
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Figure  24  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over 
double  duct  profile  range  dependent  environment  (wlanl83  extraction)  for  a  frequency  of  250  Hz. 
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Figure  25  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  surface  duct  over 
double  duct  profile  range  dependent  environment  (wlanl83  extraction)  for  a  frequency  of  2500  Hz. 


4.4  Summary  of  Deep  Water  Cases 

Three  deep  water  cases  were  presented,  the  range  independent  Munk  profile  case,  the  surface  duct  over  a 
deep  channel  case,  and  the  surface  duct  over  a  double  duct  case.  Two  rather  simple  observations  can  be 
made:  in  general  FeyRay  had  a  lower  correlation  of  error  than  did  GRAB  relative  to  a  given  threshold,  and 
FeyRay  performed  better  at  lower  frequencies  that  did  GRAB.  Both  FeyRay  and  GRAB  generated 
solutions  that  exhibited  structural  problems:  that  is  they  either  created  a  sound  channel  where  none  existed, 
or  ignored  one  which  existed. 


5.0  SHALLOW  WATER  ACCURACY  TESTS 

In  this  section  the  performance  of  FeyRay  and  GRAB  will  shown  on  three  specific  shallow  water  test  cases 
The  three  cases  are  the  ASA  benchmark  wedge,  a  range  independent  environment  with  an  upwardly 
refracting  sound  speed,  and  sea-test  results  from  impulsive  source  data  set  in  the  East  China  Sea.  Each  of 
the  first  two  data  sets  will  be  presented  for  25,  100,  250,  and  2500  Hz.  The  25  Hz  case  was  represents  the 
model’s  ability  to  stretch  into  the  low  frequency  domain;  please  note  that  GRAB’S  OAML  certification  is 
only  down  to  150  Hz.  High  frequency  cases  were  limited  to  2500  Hz  to  prevent  excessive  run  times  by 
NSPE  (which  already  required  all  weekend  to  execute).  The  East  China  Sea  data  set  will  be  presented  for 
25,  50,  100,  200,  400,  and  800  Hz  to  match  the  conditions  of  the  sea-test. 
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5.1  ASA  Wedge 

Although  not  representative  of  any  real  sediment  the  parameters  of  the  Acoustical  Society  of  America’s 
(ASA)  benchmark  wedge  predicts  bottom  loss  behavior  that  is  representative  of  the  low  loss  in  a  sediment 
covered  area.  Use  of  a  constant  sound  speed  in  the  water  column,  as  opposed  to  a  downwardly  refracting 
sound  speed,  leads  to  a  harder  propagation  case  since  beams  must  be  traced  over  longer  ranges  to  obtain  a 
reasonable  answer.  The  constant  sound  speed  in  the  water  column  is  maintained  to  avoid  interpolation 
differences  between  the  two  propagation  models.  A  shallow  water  ASA  benchmark  wedge  bottom  loss  case 
is  then  defined  by: 

•  constant  sound  speed  (1500  m/s), 

•  constant  water  depth  (200  m) , 

•  source  depth  (100  m),  receiver  depth  (50  m), 

•  sediment  sound  speed  (1700  m/s), 

•  sediment  density  (1.5  time  water  density),  and 

•  sediment  attenuation  (0.5  dB  per  wavelength). 

Transmission  loss  predictions  are  examined  at  25,  100,  250,  and  2500  Hz  for  NSPE,  GRAB  and  FeyRay. 

Figure  26  shows  the  FeyRay,  GRAB  and  NSPE  prediction  at  25  Hz.  In  general  neither  FeyRay  or  GRAB 
match  the  benchmark  solution,  misplaced  nulls  are  common  for  both  models.  Figure  27  shows  the  FeyRay, 
GRAB  and  PE  prediction  at  100  Hz.  Both  FeyRay  and  GRAB  show  better  visual  agreement  with  the 
benchmark  solution,  although  variations  are  evident.  Figure  28  shows  the  FeyRay,  GRAB  and  PE 
predictions  at  250  Hz.  As  expected,  at  higher  frequencies  the  general  level  of  the  FeyRay  and  GRAB  are 
much  close  to  the  benchmark  solution.  Again,  as  in  the  case  of  100  Hz,  there  are  clearly  misalignments  of 
peaks  and  troughs.  Figure  29  shows  the  FeyRay,  GRAB,  and  PE  predictions  at  2500  Hz.  In  this  case  the 
FeyRay,  and  GRAB  solutions  track  the  PE  solutions  quite  well.  The  FeyRay  solution  shows  more  negative 
bias  in  transmission  loss  than  does  GRAB,  with  more  obvious  interference  nulls. 

Table  15  shows  the  FeyRay  and  GRAB  correlation  error,  and  average  error  for  the  ASA  wedge  case  for  a 
60  dB  threshold.  Also  shown  is  the  maximum  detection  range  predicted  for  each  model.  The  maximum 
detection  range  is  the  range  at  which  the  predicted  signal  level  exceeds  the  threshold  level. 


Table  15.  Correlation  error,  average  error  and  maximum  detection  range  for  FeyRay  and  GRAB  for 
the  ASA  wedge  for  a  60  dB  threshold  (maximum  detection  range  is  3.35  km). 


Freq. 

FeyRay 
corr.  err 

FeyRay 

error 

GRAB 

corr  err 

GRAB 

error 

Max  range 
PE 

Max  range 
FeyRay 

Max  range 
GRAB 

25 

40% 

5.1  dB 

39% 

3.2  dB 

all 

2.4  km 

all 

100 

26% 

4.5  dB 

20% 

3.6  dB 

all 

3.2  km 

all 

250 

31  % 

4.7  dB 

37  % 

4.9  dB 

all 

all 

all 

2500 

34% 

4.8  dB 

37  % 

4.8  dB 

all 

all 

all 

Table  16  shows  the  summary  of  the  correlation  error  ,  and  average  error  for  FeyRay  and  GRAB  for  a  80  dB 
threshold.  Two  observations  can  be  made,  the  average  error  in  GRAB  is  lower  than  the  average  error  in 
FeyRay,  and  the  correlation  error  is  lower  in  FeyRay  than  in  GRAB.  GRAB  produces  a  prediction  that  is 
closer  to  the  benchmark,  but  the  shape  of  the  prediction  is  better  in  FeyRay. 
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Table  16  Correlation  error  and  average  error  for  FeyRay  and  GRAB  for  an  80  dB  threshold  for  the 
ASA  wedge  problem. 


Frequency 

FeyRay 
corr  error 

FeyRay 
average  error 

GRAB 

corr  error 

GRAB 
average  error 

25 

40% 

8.7  dB 

38  % 

3.8  dB 

100 

44% 

4.2  dB 

35  % 

1.2  dB 

250 

45  % 

3.4  dB 

51  % 

1.2  dB 

2500 

46% 

2.9  dB 

51  % 

1.1  dB 

Table  17  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  an  80  dB 
threshold.  FeyRay  and  GRAB  show  maximum  ranges  consistent  with  PE,  although  the  minimums  are  more 
variable  due  to  the  location  and  depth  of  the  first  predicted  null. 


Table  17  Minimum  and  maximum  ranges  ( in  km)  of  signal  excess  for  PE,  FeyRay,  and  GRAB  for 
the  ASA  wedge  at  80  dB  threshold  (maximum  detection  range  is  3.35  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

1.11  km 

0.465  km 

0.800  km 

all 

all 

all 

100 

1.96  km 

1.895  km 

1.005  km 

all 

all 

all 

250 

0.22  km 

1.18  km 

0.915  km 

all 

all 

all 

2500 

0.26  km 

0.875  km 

2.235  km 

all 

all 

all 

Table  18  shows  the  summary  of  the  correlation  error,  and  average  error  for  FeyRay  and  GRAB  for  a  100 
dB  threshold.  The  difference  in  the  between  the  information  in  table  2  and  table  3  is  small,  as  would  be 
expected  form  inspecting  the  transmission  loss  plots.  The  same  conclusions  for  relative  performance  drawn 
from  the  80  dB  threshold  case  hold  for  the  100  dB  threshold  case. 


31 


FeyRay  v  1.12 


March  2007 


Table  18  Correlation  error  and  average  error  for  FeyRay  and  GRAB  for  an  100  dB  threshold  for  the 
ASA  wedge  problem. 


Frequency 

FeyRay 
corr  error 

FeyRay 
average  error 

GRAB 

corr  error 

GRAB 
average  error 

25 

40  % 

8.7  dB 

39% 

3.8  dB 

100 

43  % 

4.2  dB 

35  % 

1.2  dB 

250 

45  % 

3.4  dB 

51  % 

1.2  dB 

2500 

46% 

2.9  dB 

51  % 

1.1  dB 

Table  19.  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  100  dB 
threshold.  There  is  no  significant  difference  between  the  80  dB  threshold  case  and  the  100  dB  threshold 
case. 


Table  19  Minimum  and  maximum  ranges  ( in  km)  of  signal  excess  for  PE,  FeyRay,  and  GRAB  for 
the  ASA  wedge  at  100  dB  threshold  (maximum  detection  range  is  3.35  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

1.11  km 

1.415  km 

2.715  km 

all 

all 

all 

100 

1.96  km 

1.895  km 

1.005  km 

all 

all 

all 

250 

0.22  km 

1.18  km 

0.915  km 

all 

all 

all 

2500 

0.26  km 

0.875  km 

2.235  km 

all 

all 

all 

Figure  26  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  ASA  wedge  problem 
at  a  frequency  of  25  Hz. 
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Figure  27  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  ASA  wedge  problem 
at  a  frequency  of  100  Hz. 


Figure  28  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  ASA  wedge  problem 
at  a  frequency  of  250  Hz.. 
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Figure  29  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  ASA  wedge  problem 
at  a  frequency  of  2500. 


5.2  Upwardly  Refracting  Range  Independent  Case. 

The  upwardly  refracting  case  shares  some  common  aspects  with  the  ASA  wedge  case.  The  source,  and 
receive  depths  are  the  same.  The  sound  speed  is  upwardly  refracting  with  a  sound  speed  of  1480  m/s  at  the 
surface,  and  1500  m/s  at  the  water  sediment  interface.  The  water  depth  is  a  constant  200  m,  and  the  bottom 
is  a  constant  reflection  loss  case.  The  density  is  set  to  1.5  times  that  of  water,  the  sound  speed  ratio  is  1,  and 
the  attenuation  is  set  to  0. 1  dB  per  wavelength.  The  bottom  condition  corresponds  to  a  bottom  loss  of 
approximately  13.98  dB  per  bounce.  Each  of  the  models  were  used  to  predict  transmission  loss  at  the 
standard  frequencies  out  to  a  range  of  15  km.  FeyRay,  and  GRAB  were  run  using  their  ‘default’  ray  fans. 

Figure  30  shows  the  PE,  FeyRay,  and  GRAB  predictions  for  the  upwardly  refracting  range  independent  test 
case  for  a  frequency  of  25  Hz.  Both  GRAB  and  FeyRay  over-predict  the  transmission  loss.  The  structure  of 
both  predicts  is  poor.  Figure  3 1  shows  the  PE,  FeyRay  and  GRAB  predictions  for  the  upwardly  refracting 
range  independent  test  case  for  a  frequency  of  100  Hz.  FeyRay  continues  to  over-predict  the  transmission 
loss,  while  GRAB  both  over  and  under  predicts  the  transmission  loss.  While  the  overall  structure  of  the 
transmission  loss  is  still  poor  for  both  models,  there  is  a  marked  improvement  over  the  25  Hz  case.  Figure 
32  shows  the  PE,  FeyRay  and  GRAB  predictions  for  the  upwardly  refracting  range  independent  test  case 
for  a  frequency  of  250  Hz.  Both  the  FeyRay  and  GRAB  predictions  are  much  improved,  both  in  level  and 
in  structure.  Figure  33  shows  the  PE,  FeyRay  and  GRAB  predictions  for  the  upwardly  refracting  range 
independent  test  case  for  a  frequency  of  2500  Hz.  Both  models  generally  over-predict  the  transmission  loss, 
but  the  overall  structure  of  the  loss  is  much  improved  from  a  visual  perspective. 

Table  20  shows  the  correlation  error,  average  error,  and  maximum  range  of  detection  for  FeyRay,  and 
GRAB  for  a  60  dB  threshold.  FeyRay  and  GRAB  are  comparable  in  performance  with  good  correlation 
error,  and  low  error  although  the  maximum  detection  range  predictions  are  disappointing. 
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Table  20  Correlation  error,  average  error,  and  maximum  detection  range  for  FeyRay  and  GRAB  for 
the  flat  upwardly  refracting  test  case  (maximum  detection  range  is  15  km). 


Freq. 

FeyRay 
corr.  Err 

FeyRay 

error 

GRAB 

corr  err 

GRAB 

error 

Max  range 
PE 

Max  range 
FeyRay 

Max  range 
GRAB 

25 

18  % 

2.1  dB 

16% 

2.0  dB 

4  km 

1  km 

1  km 

100 

15  % 

1.9  dB 

25  % 

2.3  dB 

6  km 

3  km 

1  km 

250 

14% 

1.7  dB 

17% 

1.8  dB 

5  km 

1  km 

2  km 

2500 

18  % 

1.9  dB 

18  % 

1.9  dB 

9  km 

5  km 

5  km 

Table  21  shows  the  summary  of  the  correlation  error  and  average  error  for  FeyRay  and  GRAB  for  a  80  dB 
threshold.  The  average  error  in  GRAB  is  lower  than  the  average  error  in  FeyRay.  The  correlation  error  in 
FeyRay  is  less  at  100  and  250  Hz  and  greater  at  25  and  2500  Hz.  Both  models,  except  GRAB  at  250  Hz 
predict  more  transmission  loss  than  is  actually  observed. 


Table  21.  Correlation  Error  and  Average  Error  for  FeyRay  and  GRAB  at  80  dB  threshold. 


Frequency 

FeyRay 

correlation  error 

FeyRay 
average  error 

GRAB 

correlation  error 

GRAB 
average  error 

25 

27  % 

7.9  dB 

21  % 

5.4  dB 

100 

27  % 

5.1  dB 

40% 

5.2  dB 

250 

37  % 

5.2  dB 

31  % 

-1.5  dB 

2500 

33  % 

5.2  dB 

35  % 

3.3  dB 

Table  22  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  an  80  dB 
threshold.  FeyRay  and  GRAB  show  maximum  ranges  consistent  with  PE,  although  the  minimums  are  more 
variable  due  to  the  location  and  depth  of  the  first  predicted  null. 


Table  22.  Minimum  and  maximum  ranges  (in  km)  of  signal  excess  for  PE,  FeyRay,  and  GRAB  for 
the  flat  upwardly  refracting  at  80  dB  threshold  (maximum  detection  range  is  15  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

3.345  km 

2.88  km 

9.475  km 

15  km 

14.995  km 

14.59  km 

100 

0.49  km 

6.435  km 

0.535  km 

15  km 

14.945  km 

15  km 

250 

1.84  km 

1.48  km 

14.25  km 

15  km 

15.0  km 

14.995  km 

2500 

1.84  km 

0.96  km 

1.49km 

15  km 

14.885  km 

15.0  km 

Table  23  shows  the  summary  of  the  correlation  error,  and  average  error  for  FeyRay  and  GRAB  for  a  100 
dB  threshold.  The  difference  in  the  between  the  information  in  table  2  and  table  3  is  small,  as  would  be 
expected  form  inspecting  the  transmission  loss  plots.  The  same  conclusions  for  relative  performance  drawn 
from  the  80  dB  threshold  case  hold  for  the  100  dB  threshold  case. 


Table  23.  Correlation  Error  and  Average  Error  for  FeyRay  and  GRAB  at  100  dB  threshold. 


Frequency 

FeyRay 
corr  error 

FeyRay 
average  error 

GRAB 

corr  error 

GRAB 
average  error 

25 

32% 

9.4  dB 

21  % 

5.6  dB 

100 

31  % 

5.7  dB 

41  % 

5.6  dB 

250 

44% 

5.6  dB 

34% 

-1.8  dB 

2500 

49% 

5.9  dB 

42% 

3.5  dB 

Table  24.  shows  the  minimum  and  maximum  range  of  detection  for  PE,  FeyRay,  and  GRAB  for  100  dB 
threshold.  There  is  no  significant  difference  between  the  80  dB  threshold  case  and  the  100  dB  threshold 
case.  There  is  tendency  for  GRAB  to  over-predict  (i.e.  range  longer  than  PE  prediction)  the  minimum  range 
at  which  the  signal  falls  below  the  threshold. 
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Table  24.  Minimum  and  maximum  ranges  ( in  km)  of  signal  excess  for  PE,  FeyRay,  and  GRAB  for 
the  flat  upwardly  refracting  at  100  dB  threshold  (maximum  detection  range  is  15  km). 


Frequency 

PE  min 

FeyRay  min 

GRAB  min 

PE  max 

FeyRay  max 

GRAB  max 

25 

3.345  km 

3.745  km 

9.475  km 

15  km 

14.945  km 

15  km 

100 

0.49  km 

8.74  km 

10.58  km 

15  km 

15  km 

15  km 

250 

14.08  km 

8.095  km 

14.25  km 

15  km 

15  km 

15  km 

2500 

4.025  km 

7.55  km 

9.665  km 

15  km 

15  km 

15  km 

Range  (km) 

Figure  30  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  the  range 
independent  upwardly  refracting  case  at  a  frequency  of  25  Hz. 
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Figure  31  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  the  range 
independent  upwardly  refracting  case  at  a  frequency  of  100  Hz. 
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Figure  32  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  the  range 
independent  upwardly  refracting  case  at  a  frequency  of  250  Hz. 
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Figure  33  .  Plot  of  transmission  loss  predictions  for  PE,  FeyRay,  and  GRAB  for  the  range 
independent  upwardly  refracting  case  at  a  frequency  of  2500  Hz.. 


5.3  East  China  Sea  Measured  Data 

The  East  China  Sea  case  is  one  of  two  cases  (the  other  being  Lloyd’s  mirror)  where  the  benchmark  is  not 
PE.  In  this  case  the  benchmark  is  measured  transmission  loss  taken  in  octave  bands  ranging  from  25  Hz  to 
800  Hz.  The  sound  speed  environment,  and  bathymetry  are  depicted  in  Figure  34.  The  bottom  loss 
parameters  were  obtained  from  inverting  the  transmission  loss  data  using  a  simulated  annealing  method, 
assuming  a  two  layer  bottom.  The  results  were  a  surface  sediment  of  velocity  ratio  1.12,  density  1.8,  and 
attenuation  0.4  dB  per  wavelength  of  depth  40  m,  overlying  a  sediment  of  sound  speed  ratio  1.2,  density  2, 
and  attenuation  0.5  dB  per  wavelength.  For  these  model  runs  the  sediment  was  set  as  a  Rayleigh  bottom 
with  velocity  ratio  of  1.12,  density  of  1.8  and  attenuation  of  0.4  dB  per  meter.  The  model  output  are  range 
averaged  over  350  m,  and  interpolated  to  the  measured  data  point  locations. 

Figure  35  shows  the  measured  transmission  loss,  FeyRay  prediction,  and  GRAB  prediction  for  25  Hz  for 
the  environment  depicted  in  Figure  20.  The  models  follows  the  measured  transmission  loss  quite  well  for 
the  entire  range,  both  models  exhibit  a  maximum  excursion  of  10  dB.  Figure  36  shows  the  measured 
transmission  loss,  the  FeyRay  prediction  ,  and  GRAB  prediction  for  50  Hz  for  the  environment  depicted  in 
Figure  20.  Again,  the  predictions  are  quite  reasonable  renditions  of  the  measured  data. 

Figure  37  shows  the  measured  transmission  loss,  the  FeyRay  prediction,  and  GRAB  prediction  for  100  Hz 
for  the  environment  depicted  in  Figure  20.  The  100  Hz  prediction  by  GRAB  is  better  than  the  100  Hz 
FeyRay  prediction.  The  GRAB  prediction  much  more  closely  follows  the  level  of  the  data.  Since  the  data  is 
largely  featureless,  tracking  features  is  not  an  issue.  Figure  38  shows  the  measured  transmission  loss, 
FeyRay  prediction,  and  GRAB  prediction  for  200  Hz.  Both  models  again  produce  excellent  predictions 
selecting  one  model  over  the  other  based  on  visual  evidence  would  be  difficult.  Figure  39  shows  the 
measured  transmission  loss,  the  FeyRay  prediction,  and  the  GRAB  prediction  for  400  Hz.  Both  models 
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produce  very  acceptable  predictions,  with  FeyRay  visually  appearing  the  more  accurate.  Figure  40  shows 
the  measured  transmission  loss,  FeyRay  prediction,  and  GRAB  prediction  for  800  Hz.  Both  models  are 
under-predict  the  transmission  loss,  since  the  error  is  systematic  some  of  the  difference  is  probably  due  to 
an  over-simplistic  inversion  model.  There  is  a  slight  upward  refracting  layer  in  the  sound  speed  column. 
Each  model  fits  differently  which  may  account  for  some  or  most  of  the  difference 

Table  25  shows  the  correlation  error,  average  error  and  maximum  detection  range  for  FeyRay,  and  GRAB 
for  a  60  dB  threshold,  also  show  is  the  maximum  detection  range  as  measured.  The  ranges  are  show  in  nmi 
rather  than  km,  as  the  spacing  of  the  measurement  buoys  was  determined  using  this  measurement  system. 


Table  25  Correlation  error,  average  error  and  maximum  detection  range  for  FeyRay,  and  GRAB  for 
the  East  China  Sea  Test  Case  (maximum  possible  range  is  31  nmi). 


Freq. 

FeyRay 
corr.  Err 

FeyRay 

error 

GRAB 

corr  err 

GRAB 

error 

Max  range 
measured 

Max  range 
FeyRay 

Max  range 
GRAB 

25 

1  % 

0.2  dB 

3  % 

1.4  dB 

2  nmi 

2  nmi 

2  nmi 

50 

12% 

1.0  dB 

2% 

0.9  dB 

3  nmi 

1  nmi 

2  nmi 

100 

8  % 

1.5  dB 

5  % 

2.6  dB 

3  nmi 

2  nmi 

2  nmi 

200 

2% 

2.1  dB 

10% 

2.6  dB 

2  nmi 

2  nmi 

3  nmi 

400 

3  % 

2.2  dB 

3  % 

2.6  dB 

2  nmi 

2  nmi 

4  nmi 

800 

1  % 

2.5  dB 

7  % 

3.3  dB 

1  nmi 

2  nmi 

3  nmi 

Table  26  shows  the  correlation  error,  and  average  error  for  FeyRay  and  GRAB  for  a  80  dB  threshold.  The 
statistics  in  this  table  demonstrate  that  the  visual  evidence  seen  in  figures  21  through  26  are  reflected  in  a 
quantifiable  manner.  The  GRAB  correlation  error  is  roughly  twice  that  of  the  FeyRay  correlation  error,  and 
the  maximum  average  error  for  either  model  is  3.8  dB. 


Table  26  Correlation  Error  and  average  error  for  FeyRay  and  GRAB  for  the  East  China  Sea 
measured  data  case  for  80  dB  threshold. 


Frequency 

FeyRay  correlation 
error 

FeyRay  average 
error 

GRAB  correlation 

error 

GRAB  average 
error 

25 

5  % 

1.0  dB 

10% 

-1.2  dB 

50 

6% 

-1.4  dB 

10% 

-1.0  dB 

100 

7  % 

-2.5  dB 

8% 

-0.2  dB 

200 

5  % 

-1.6  dB 

11  % 

1.1  dB 

400 

4% 

1.5  dB 

10% 

3.1  dB 

800 

3  % 

3.4  dB 

6% 

3.8  dB 

Table  27  shows  the  minimum  and  maximum  detection  ranges  for  measured  data,  FeyRay,  and  GRAB  for  a 
80  dB  threshold.  Both  models  produce  acceptable  results,  which  some  overestimate  of  minimum  range  and 
some  underestimate. 
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Table  27.  Minimum  and  maximum  range  of  detection  for  the  measured  data,  FeyRay,  and  GRAB  for  the 


East  China  Sea  test  case  for  80  dB  threshold  (maximum  detection  range  is  31  nmi). 


Frequency 

Measured 

min 

FeyRay  min 

GRAB  min 

Measured 

max 

FeyRay  max 

GRAB  max 

25 

16  nmi 

12  nmi 

14  nmi 

16  nmi 

18  nmi 

28  nmi 

50 

18  nmi 

9  nmi 

13  nmi 

18  nmi 

18  nmi 

18  nmi 

100 

24  nmi 

13  nmi 

18  nmi 

24  nmi 

18  nmi 

18  nmi 

200 

25  nmi 

9  nmi 

27  nmi44 

25  nmi 

18  nmi 

27  nmi 

400 

19  nmi 

9  nmi 

17  nmi 

19  nmi 

1 1  nmi 

28  nmi 

800 

1 1  nmi 

9  nmi 

14  nmi 

1 1  nmi 

1 1  nmi 

27  nmi 

Sound  Speed  (m/s) 

1500  1510  1520  1530  1540  1550 


0  10  20  30  40  50  60 

Range  (km) 

Figure  34  .  Plot  of  the  bathymetry,  and  representative  sound  speed  profile  for  the  East  China  Sea 
measured  data  case. 
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Figure  35  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  25  Hz  for  the  East  China  Sea  data  case. 
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Figure  36  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  50  Hz  for  the  East  China  Sea  data. 
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Figure  37  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  100  Hz  for  the  East  China  Sea  data. 


44 


March  2007 


FeyRay  v  1.12 


Figure  38  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  200  Hz  for  the  East  China  Sea  data  case. 
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Figure  39  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  400  Hz  for  the  East  China  Sea  data  case. 
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Figure  40  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  800  Hz  for  the  East  China  Sea  data  case. 


5.4  Summary  of  Shallow  Water  Cases 

Neither  FeyRay  nor  GRAB  showed  stellar  performance  against  the  ASA  wedge,  or  the  upwardly  refracting 
range  independent  test  case.  In  general  FeyRay  had  a  better  shape,  as  revealed  by  the  correlation  error,  and 
GRAB  had  a  better  level.  It  seems  as  though  GRAB  does  a  better  job  of  predicting  the  average  level  over 
range,  but  misses  the  structure  of  the  transmission  loss.  FeyRay  seems  to  predict  the  structure  better,  but  the 
level  is  slightly  worse  than  GRAB.  Against  the  measured  East  China  Sea  data  both  models  performed  very 
well.  FeyRay  seemed  to  slightly  outperform  GRAB  in  this  case,  but  either  model  prediction  is  very  good. 
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6.0  CONCLUSIONS 

The  deep  water  and  shallow  water  accuracy  test  are  summarized  in  Figures  41  and  42.  These  summaries 
illustrate  the  average  error  values  across  frequencies  for  the  100  dB  threshold  cases.  Figure  41  represents 
the  mean  and  standard  deviation  of  the  “average  difference  in  level’'  metric.  Figure  42  represents  the  mean 
and  standard  deviation  of  the  “average  correlation  error"  metric.  The  results  are  presented  as  a  bar  chart  in 
which  the  performance  of  GRAB  and  FeyRay,  relative  to  a  reference  solution  (NSPE  or  sea  test 
measurements),  is  illustrated  for  each  of  the  deep  water  and  shallow  water  accuracy  tests. 


Comparisons  to  PE  (100  dB  Threshold) 


□  FeyRay 
KGRAB 


Figure  41  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  800  Hz  for  the  East  China  Sea  data  case. 
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Comparisons  to  PE  (100  dB  Threshold) 


□  FeyRay 
KGRAB 


Figure  42  .  Plot  of  transmission  loss  measured,  FeyRay  prediction,  and  GRAB  prediction  for  a 
frequency  of  800  Hz  for  the  East  China  Sea  data  case. 


In  general,  the  accuracy  tests  show  that  the  level  of  GRAB  is  slightly  closer  to  the  reference  than  the 
corresponding  FeyRay  results.  But,  the  correlation  error  of  the  FeyRay  results  are  slightly  lower  than  their 
GRAB  counterparts.  However,  the  error  bars  in  Figures  41  and  42  indicate  that  these  differences  are  not 
statistically  significant.  Figures  4-11  illustrate  that  a  similar  conclusion  of  equivalence  can  be  drawn  for  the 
random  environments  sampled  in  the  robustness  tests.  According  to  these  measurements,  the  propagation 
loss  values  computed  by  FeyRay  are  statistically  equivalent  in  both  level  and  structure  to  the  values 
computed  by  the  GRAB  model. 

The  timing  tests  reveal  that  the  execution  speed  of  FeyRay  is  significantly  better  than  that  of  GRAB.  Even 
after  eliminating  differences  in  the  time  needed  to  load  the  program  into  memory  and  GRAB’S  heavier  use 
of  disk  I/O,  the  FeyRay  calculations  ran  approximately  20  times  faster  than  GRAB  results  of  the  same 
accuracy.  Recent  changes  have  been  made  to  GRAB  to  improve  its  speed  (at  the  cost  of  accuracy),  but 
FeyRay  was  still  5  times  faster  than  the  improved  GRAB  model. 

In  my  judgment,  these  measurements  indicate  that  the  accuracy  and  execution  speed  of  FeyRay  are 
sufficient  to  justify  its  use  in  high-fidelity  training  systems. 


7.0  REFERENCES 

Porter,  Michael  B.  and  Homer  P.  Bucker,  1987.  “Gaussian  beam  tracing  for  computing  ocean  acoustic 
fields”,  JAS A  Vol  82  no.  4,  p.  1349-1359. 

Foreman,  Terry  L..  “The  FeyRay  Gaussian  Beam  Underwater  Acoustic  Propagation  Model”,  Analysis  and 
Technology  Washington  Operations  Center  Washington  DC  {unpublished  manuscript,  2007} 


49 


FeyRay  v  1.12 


March  2007 


Munk,  H.,  1974,  “  Sound  channel  in  an  exponentially  stratified  ocean  with  applications  to  SOFAR,”  JASA 
Vol  76  220-226. 


50 


